Evolving an API Design

What is API evolution? #

API evolution refers to the process of making changes to an API over time while maintaining compatibility with existing client applications. This involves introducing new features, deprecating old ones, and making other modifications that enhance the functionality of the API, without causing any disruption to its users. The goal of API evolution is to keep the API current and relevant, while ensuring that it remains usable by existing clients.

In this lesson, we’ll take a look at some approaches that may help us evolve our API.

Changing data access patterns#

As an API gets popular, the consumers of the API might begin to use it in a way that was not initially anticipated by the designers.

Let's take the example of Twitter. A few years ago, the Twitter application could only receive new tweets in real-time by constantly polling the back-end API. This eventually led to the REST APIs getting a huge increase in traffic and caused scalability issues for the rapidly expanding user base.

So how did Twitter counteract this? In August 2020, they launched a new streaming API, which sends new data and reduces polling. This streaming API allowed developers to subscribe to selected keywords or users and, in turn, allowed them to receive new tweets over a persistent connection.

Note: Event-driven architecture is an important concept and we discuss it in detail later on in this course.

Adding new API methods#

Sometimes we may have APIs that are essential, but are resource expensive to call and have huge response sizes. Developers don't always need all the data and may only need a small subset of the responses to meet their requirements. If the only option that the developers have is to request all the data or none of the data, the developers might end up getting full responses, and most of the payload might go ignored by the developers who only require a smaller amount of the data. The payload that’s being ignored might have been expensive to compute, and if it isn’t needed, then it’s a waste of precious resources.

Making changes to an API once it has been published is not an easy task, but adding a new method is relatively easy. Let’s take the example of Slack. Slack had a very popular API method for its Real Time Messaging (RTM) API called rtm.start. It would start an RTM session and return a lot of data regarding things like teams, channels, members, and so on. This method was initially designed for small teams, but as team sizes began to grow, the payload size became a problem for the consumers of the API. Most consumers were only using this method to establish connections to the WebSocket. Keeping this in mind, the Slack team released a new API method called rtm.connect. This method would only return data relevant to the WebSocket session and helped alleviate the scaling and resource wastage problems being caused by the rtm.start method.

Providing options to filter the results#

If an API returns a number of objects, it’s imperative to consider providing options for filtering the results. This gives developers the freedom to limit the number of objects being returned by the API to suit their needs. Adding these options makes our API more scalable. The table below consists of common API filters.

Supporting bulk endpoints#

In some cases, developers must do the same operation on several items, such as adding multiple users to a group chat. This may require sending individual API calls for each operation and can lead to increased latency for the client. This also might be a problem in systems of a large scale. To circumvent this, we can support bulk endpoints. Bulk endpoints refer to combining multiple calls of the same type into an array and executing them as a single request. Bulk endpoints are more efficient because they require fewer HTTP round trips.

Let’s once again take Slack as an example. In the past, inviting multiple users to a Slack channel would require developers to call the channels.invite method for each individual user. To improve this, Slack added support for inviting multiple people in a single API call, thereby saving costs for Slack and the developers. An example of this is given in the code snippet below:

In the code snippet above, we can see that the invite endpoint has only been called once, and we’re able to add multiple users through a single call, instead of individually calling the endpoint for each user.

Let's take a minute to review everything we’ve learned so far:

Summary#

As we can see, scaling APIs is inevitable. Our API design needs to be flexible and evolvable so that the scaling process is easier in the future. Not only does evolving the design help with scaling, but it also provides our consumers with a robust and resilient design and gives them options to implement the API according to their needs.